Are male and female jackal mandible lengths different?
Mandible lengths of female and male jackals from the Natural History Museum (London).
n = 20; Difference of means = -4.8
Are male and female jackal mandible lengths different?
How big of a difference can happen with random sampling in our dataset?
How big of a difference can happen with random sampling in our dataset?
How big of a difference can happen with random sampling in our dataset?
Difference in means for this sample: 2
Sample over and over
Sample over and over
Observed difference = -4.8
Proportion of randomized differences more extreme than the observed
Mean of the differences where the value is less than or equal to the observed mean difference.
Empirically determined P-value is 0.0018.
The expectation from random sampling underlies our statistical tests
t.test(Mandible ~ Sex, data = M)
Welch Two Sample t-test
data: Mandible by Sex
t = -3.4843, df = 14.894, p-value = 0.00336
alternative hypothesis: true difference in means between group F and group M is not equal to 0
95 percent confidence interval:
-7.738105 -1.861895
sample estimates:
mean in group F mean in group M
108.6 113.4
Why would you need Monte Carlo Methods?
Your Data Set Violates the Assumption(s) of Parametric Tests
Very common issue
Many Monte Carlo methods are “distribution-free”
Monte Carlo almost always preferred to a “non-parametric test”
Sign test
Mann-Whitney U (Wilcoxon rank-sum)
Kruskal-Wallace
There is not a standard way to estimate a confidence interval
Our method of estimating confidence intervals assumes a particular distribution
e.g., what is the CI on a proportion?
You aren’t sure the analysis you are doing is OK
You are using an analysis or the results in a non-typical way
You want to know the rate of decision errors
false positives
false negatives
Your question isn’t answered by a typical statistical test
Groups can be different due to multiple causes only some of which are of interest
Standard statistical tests account only for sampling error in the null
Why would you need Monte Carlo Methods?
Monte Carlo Methods provide a toolkit to ask very practical questions
Do simulation experiments to find out what happens